A number of applications exist in which it is desired to collect crowd-based scoring of a diverse set of choices. For example, opinion polls can be used to develop predictions for outcomes of sporting events and elections; prediction markets can be used to derive forecasts based on individual behavior expressed through social games, trading of actual or virtual stocks, and other mechanisms. Yet another exemplary application includes systems that make recommendations for movies, music, books and other content, based on tracking the past behavior of individuals or certain demographics and sometimes based on soliciting preferences (“votes”) between hypothetical choices. Many other applications exist. Irrespective of application, these systems are often manifested as software running on a computer (e.g., standalone computer, portable device, smart phone, server, diverse network machines or some other form of one or more instructed-machines). This software performs automated polling or collection of data, and scoring of results.
While generally useful for their intended purposes, these systems generally are poor at extracting group preferences as the number of possible choices increases. For example, each individual in a group can be asked to order available choices as a set or to vote between each possible pair of choices. However, some individuals are daunted by the time needed to order a large set of choices or to vote between many combinations, and therefore do not participate. Some individuals do not take the time to view and evaluate each choice before voting, and so, express their results in a biased manner. It can be computationally intensive to collect preference data between each possible pair of choices. Finally, some individuals vote on some choices (e.g., their favorites) but not all possible choices, rendering it difficult to interpret relative scores where the voting weights are different. The consequence of these issues is that the mentioned-systems can produce results based on a sparse, biased subset of group beliefs, and it can become difficult to generate a comprehensive, accurate scoring across all possible choices.
What is needed are systems, machines, software and techniques that address the aforementioned problems. The present invention satisfies these needs and provides further, related advantages.
The subject matter defined by the enumerated claims may be better understood by referring to the following detailed description, which should be read in conjunction with the accompanying drawings. This description of one or more particular embodiments, set out below to enable one to build and use various implementations of the technology set forth by the claims, is not intended to limit the enumerated claims, but to exemplify their application. The description exemplifies methods and devices, especially as software or a network service, supporting use of a height-balanced tree to govern the eliciting of relative preference data between discrete subsets of an overall set, with the result that relative preference data scoring need not be obtained between every possible subset combination of members of a large set. While the specific examples are presented, the principles described herein may also be applied to other methods, devices and systems as well.